Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(starknet_l1_provider): add scraper #2853

Merged
merged 1 commit into from
Jan 6, 2025
Merged

Conversation

giladchase
Copy link
Contributor

@giladchase giladchase commented Dec 20, 2024

  • base layer constant file will hold the identifiers of Starknet Events.
  • fetch_events will get events from base layer and send to provider

@reviewable-StarkWare
Copy link

This change is Reviewable

@giladchase giladchase changed the base branch from main to gilad/add-base-layer-to-infra December 20, 2024 20:45
@giladchase giladchase changed the title Gilad/add scraper feat(starknet_l1_provider): add scraper Dec 20, 2024
Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 7 of 7 files at r1, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @giladchase and @matan-starkware)


crates/starknet_l1_provider/src/l1_scraper.rs line 55 at r1 (raw file):

            self.fetch_events().await?;
        }
    }

Is this the canonical run implementation in our node?

Code quote:

    async fn _run(&mut self) -> L1ScraperResult<(), B> {
        loop {
            sleep(self.config.polling_interval).await;
            // TODO: retry.
            self.fetch_events().await?;
        }
    }

crates/starknet_l1_provider/src/l1_scraper.rs line 60 at r1 (raw file):

#[derive(Clone, Debug, Serialize, Deserialize, Validate, PartialEq)]
pub struct L1ScraperConfig {
    // TODO: make this config into trait.

WDYM? Why?

Code quote:

// TODO: make this config into trait.

@giladchase giladchase force-pushed the gilad/add-base-layer-to-infra branch from f887cbb to 83be466 Compare December 24, 2024 12:03
Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 2 files at r2, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @giladchase and @matan-starkware)

Copy link
Contributor

@matan-starkware matan-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 7 files at r1, 2 of 2 files at r2, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @giladchase)


-- commits line 4 at r2:
We don't want these to be configurable?

Code quote:

  - base layer constant file will hold the identifiers of Starknet Events.

crates/starknet_l1_provider/src/l1_scraper_tests.rs line 0 at r2 (raw file):
Why add an empty file?

@giladchase giladchase force-pushed the gilad/add-base-layer-to-infra branch from 83be466 to 1c38ffc Compare December 25, 2024 12:30
Copy link
Contributor Author

@giladchase giladchase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 5 of 8 files reviewed, 3 unresolved discussions (waiting on @elintul and @matan-starkware)


-- commits line 4 at r2:

Previously, matan-starkware wrote…

We don't want these to be configurable?

I might be misunderstanding you, but AFAIK event identifiers (in Ethereum) are fixed, they are always:
keccak(<signature_of_event>).
The identifier will only change if the signature changes, and this updates the const during compilation time (since the consts are generated from sol!).


crates/starknet_l1_provider/src/l1_scraper.rs line 55 at r1 (raw file):

Previously, elintul (Elin) wrote…

Is this the canonical run implementation in our node?

I haven't found any similar run use-case in our code. We currently only have loop-and-select! style runs, without any loop-do-sleep-awake loops.

With that said, this is a very basic implementation, I'll extend it soon once the pressure's off, unless you have stuff in mind now?


crates/starknet_l1_provider/src/l1_scraper.rs line 60 at r1 (raw file):

Previously, elintul (Elin) wrote…

WDYM? Why?

Seems to be refactoring leftovers, doesn't make sense here, removed.

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 3 of 3 files at r3, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @giladchase and @matan-starkware)


crates/starknet_l1_provider/src/l1_scraper.rs line 55 at r1 (raw file):

Previously, giladchase wrote…

I haven't found any similar run use-case in our code. We currently only have loop-and-select! style runs, without any loop-do-sleep-awake loops.

With that said, this is a very basic implementation, I'll extend it soon once the pressure's off, unless you have stuff in mind now?

Nope, sounds good.

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @giladchase and @matan-starkware)

Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @matan-starkware)

@giladchase giladchase force-pushed the gilad/add-base-layer-to-infra branch 2 times, most recently from 258b7d5 to 5d94a73 Compare January 2, 2025 04:06
@giladchase giladchase changed the base branch from gilad/add-base-layer-to-infra to spr/main/886f2b3e January 2, 2025 08:08
@giladchase giladchase closed this Jan 2, 2025
@giladchase giladchase reopened this Jan 2, 2025
Copy link
Collaborator

@elintul elintul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r4, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @matan-starkware)

Copy link
Contributor

@matan-starkware matan-starkware left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 7 files at r1, 2 of 3 files at r3, 1 of 1 files at r4, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @giladchase)


-- commits line 4 at r2:

Previously, giladchase wrote…

I might be misunderstanding you, but AFAIK event identifiers (in Ethereum) are fixed, they are always:
keccak(<signature_of_event>).
The identifier will only change if the signature changes, and this updates the const during compilation time (since the consts are generated from sol!).

I understand a given event has a fixed identifier, I was wondering if the set of identifiers we want to track may change over time or with tests.

Ok thanks for explaining about them being generated by sol!. I see in your later PR pub const LOG_MESSAGE_TO_L2_EVENT_IDENTIFIER: &str = Starknet::LogMessageToL2::SIGNATURE; So my idea was infeasible

@Itay-Tsabary-Starkware Itay-Tsabary-Starkware force-pushed the spr/main/886f2b3e branch 2 times, most recently from 6ef9323 to d46875d Compare January 5, 2025 11:08
@giladchase giladchase changed the base branch from spr/main/886f2b3e to main January 5, 2025 15:28
@giladchase giladchase force-pushed the gilad/add-scraper branch 2 times, most recently from dbf8284 to f222103 Compare January 6, 2025 07:28
- base layer constant file will hold the identifiers of Starknet Events.
- fetch_events will get events from base layer and send to provider
Copy link
Contributor Author

@giladchase giladchase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 5 of 8 files reviewed, all discussions resolved (waiting on @elintul and @matan-starkware)


-- commits line 4 at r2:

I understand a given event has a fixed identifier, I was wondering if the set of identifiers we want to track may change over time or with tests.

For sure, that is covered by the tracked_event_identifiers field which is passed to the provider in its constructor.

Copy link
Contributor Author

@giladchase giladchase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 7 files at r1, 1 of 2 files at r2, 1 of 3 files at r3, 3 of 3 files at r5, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @giladchase)

@giladchase giladchase added this pull request to the merge queue Jan 6, 2025
Merged via the queue into main with commit d75a79f Jan 6, 2025
21 checks passed
@giladchase giladchase deleted the gilad/add-scraper branch January 6, 2025 09:17
@github-actions github-actions bot locked and limited conversation to collaborators Jan 8, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants